Geometrical Initialization, Parametrization and Control of Multilayer Perceptrons: Application to Function Approximation

This paper proposes a new method to reduce training time for neural nets used as function approximators. This method relies on a geometrical control of Multilayer Perceptrons (MLP). A geometrical initialization gives first better starting points for the learning process. A geometrical parametrization achieves then a more stable convergence. During the learning process, a dynamic geometrical control helps to avoid local minima. Finally, simulation results are presented, showing drastic reduction in training time and increase in convergence rate. I. Introduction Mathematical theorems prove the existence of a twolayer perceptron, with sigmoidal units in the hidden layer, that approximate a given real valued multivariate continuous function to a given degree of accuracy [3, 5, 8]. Such approximation methods are very important for pratical purposes : they can be used, for instance, in black-box identification and control of nonlinear systems, or in time series prediction. The main problem..

NSK, an Object-Oriented Simulator Kernel for Arbitrary Feedforward Neural Networks

An object-oriented neural network simulator kernel is presented. It is based on a general mathematical model for arbitrary feedforward nets. We propose a C++ implementation of this model which satisfies the following requirements : expandability (allowing an easy implementation of a new neural model), portability and efficiency (the kernel does not increase significantly computation times for classic models, compared to a direct object-oriented implementation). Learning algorithms such as gradient-based ones can be written for arbitrary nets and are therefore directly available for every particular model. 1 Introduction Due to the lack of mathematical theories about the capabilities of neural networks, the discovery of interesting properties of these networks strongly relies on computer simulations. One of the main practical problems we have to deal with when trying to experiment a new network model is the development of an implementation of this model in order to conduct experimenta..